Crowd workers proven useful: A comparative study of subjective video quality assessment
نویسندگان
چکیده
We carried out crowdsourced video quality assessments using paired comparisons and converting the results to differential mean opinion scores (DMOS). A previous labbased study had provided corresponding MOS-values for absolute category ratings. Using a simple linear transformation to fit the crowdsourcing-based DMOS values to the lab-based MOS values, we compared the results in terms of correlation coefficients and visually checked the relationship on scatter plots. The comparison result is surprisingly good with correlation coefficients more than 0.96, although (1) the original video sequences had to be cropped and downscaled in the crowdsourcing-based experiments, (2) the control of the experimental setup for the crowdsourcing case was much less and (3) it was widely believed that data from crowdsourcing workers are less reliable. Our result suggests crowdsourcing workers can actually be used to collect reliable VQA data in some applications.
منابع مشابه
A Machine Learning Approach to No-Reference Objective Video Quality Assessment for High Definition Resources
The video quality assessment must be adapted to the human visual system, which is why researchers have performed subjective viewing experiments in order to obtain the conditions of encoding of video systems to provide the best quality to the user. The objective of this study is to assess the video quality using image features extraction without using reference video. RMSE values and processing ...
متن کاملQualityCrowd - A framework for crowd-based quality evaluation
Video quality assessment with subjective testing is both time consuming and expensive. An interesting new approach to traditional testing is the so-called crowdsourcing, moving the testing effort into the internet. We therefore propose in this contribution the QualityCrowd framework to effortlessly perform subjective quality assessment with crowdsourcing. QualityCrowd allows codec independent q...
متن کاملA Comparative Study of Gender and Age Classification in Speech Signals
Accurate gender classification is useful in speech and speaker recognition as well as speech emotion classification, because a better performance has been reported when separate acoustic models are employed for males and females. Gender classification is also apparent in face recognition, video summarization, human-robot interaction, etc. Although gender classification is rather mature in a...
متن کاملSubjective Quality Assessment of H.264/AVC Video Streaming with Packet Losses
Research in the field of video quality assessment relies on the availability of subjective scores, collected by means of experiments in which groups of people are asked to rate the quality of video sequences. The availability of subjective scores is fundamental to enable validation and comparative benchmarking of the objective algorithms that try to predict human perception of video quality by ...
متن کاملMobile multimedia crowd service cooperation control protocol based on opportunistic wavelet
In order to improve the quality of multimedia communication and experience, we proposed a crowd service cooperation control protocol based on the opportunistic wavelet model. Firstly, in order to provide real-time and reliable guarantee for mobile multimedia services, we analyze the delay, jitter, and distortion of wireless multimedia streams transmission based on wavelet model. The method of v...
متن کامل